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DETAILED ACTION 



Claim Objections 



Claim 10 is objected to because of the following informalities: the claim terminates 
improperly. Appropriate correction is required. 

Allowable Subject Matter 

Claims 10-13, 17 and 21 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for 
patent or (2) a patent granted on an application for patent by another filed in the United 
States before the invention by the applicant for patent, except that an international 
application filed under the treaty defined in section 351(a) shall have the effects for 
purposes of this subsection of an application filed in the United States only if the 
international application designated the United States and was published under Article 
21(2) of such treaty in the English language. 

Claims 1-7, 14-16, 18, and 22-26 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Edwards et al (U.S. Patent 6,459,442), hereinafter Edwards. 

Regarding claim 1, Edwards shows inputting at least one object on a screen display, 
taught as "segments" which have stroke behaviors applied to them, at cols. 4-5, lines 62-67 and 
1-6. Edwards further teaches using an input device to draw at least one arrow having a 
configuration recognized by software to associate the arrow with at least one object, and having 
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an arrow convey a transaction relating to said at least one object, taught as the painting, 
recognition, and implementation of an arrow "shortcut" gesture, in Fig. 5, col. 6, lines 2-5 and 
14-16. 

Regarding claim 2, Edwards teaches an arrow including a tail end (by definition arrows 
have a head end and a tail end), where a configuration comprises at least one object being 
disposed within a definable distance to the tail end, taught as the association of a stroke with a 
segment due to a predefined distance between the two, at col. 6, lines 56-62. 

Regarding claim 3, Edwards teaches a further object disposed within a definable 
distance to the head end of said arrow, with a transaction being directed by the arrow to be 
carried out from at least one object to the further object, taught as the use of a drawing of a line 
to "join" segments on a display, at col. 10, lines 36-40. Although Edwards claims a straight line 
as the "join" stroke, drawing a line and drawing an arrow are sufficiently similar in view of the 
claim language to qualify as prior art. 

Regarding claim 4, Edwards teaches a selecting a transaction from a set of definable 
transaction categories, each transaction category having associated with it a definable arrow 
appearance, taught as the difference between "join" and "split" segment operations, at col. 10, 
lines 36-49. 

Regarding claims 5-7, Edwards teaches an arrow appearance including at least one of a 
plurality of color choices, plurality of line styles, or combination of the two, each corresponding 
to at least one of said transaction categories, taught as the use of context identifiers to store 
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such information as stroke color and location, and inherently stroke shape, or style (see Fig. 5). 
Each stroke is identified with a "behavior", or transaction category. See col. 8, lines 40-56. 

Regarding claim 14, Edwards teaches at least one object comprising a previous arrow 
drawn from an on-screen object, the at least one arrow extending to a portion of a previous 
arrow, with a modifier command entered within a definable distance to said at least one arrow, 
the at least one arrow acting to apply the modifier command to the transaction conveyed by the 
previous arrow, taught as the modification of a set of painted strokes, at col. 12, lines 7-10. 

Regarding claim 15, Edwards teaches entering a modifier command as text adjacent to 
said at least one arrow, taught as the calculation of handwritten formulas in response to the user 
performing the line stroke, at col. 12, lines 18-24. 

Regarding claim 16, Edwards shows a previous arrow drawn from an on-screen object, 
the at least one arrow having a front end extending to a portion of a previous arrow, and a tail 
end associated with another on-screen object, the at least one arrow acting to apply the function 
of another on-screen object to the transaction conveyed by the previous arrow, taught as the 
use of multiple behaviors applied to a segment, at col. 1 1 , lines 17-25. 

Regarding claim 18, Edwards teaches copying said at least one object to an on-screen 
location adjacent to the front end of at least one arrow, taught as the use of a "move" gesture for 
repositioning segments, at col. 10, lines 27-29. 
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Regarding claim 22, Edwards teaches changing said at least one object to equal at least 
one attribute of a further on-screen object disposed within a user definable distance to the front 
end of at least one arrow, taught as the erasure of a two-stroke street by performing the erase 
gesture on one stroke of the street, at col. 13, lines 29-32. 

Regarding claim 23, Edwards shows at least one arrow adapted to change its aesthetic 
properties after being drawn and recognized, and further including the step of touching the 
arrow with an input device to implement said transaction, taught as the representation in 
multiple colors and subsequent user selection of candidate strokes, at col. 12, lines 31-41. 

Regarding claim 24, Edwards teaches entering a Show Arrow command to make visible 
at least one arrow previously drawn and disappeared, taught as the use of a menu to recover 
stored strokes, segments, and behavior history, at col. 9, lines 19-21. 

Regarding claim 25, the Show Arrow command comprises a hand drawn symbol 
recognized by the software as a Show Arrow command, taught as allowing access to the stored 
stroke menu after the recognition of a tap gesture, at col. 6, lines 6-12. 

Regarding claim 26, Edwards teaches at least one object including a variable controller, 
and a transaction includes specifying the direction of movement of the controller to increase and 
decrease said variable, taught as the use of a time slider to specify a point in time in the user's 
stroke history, at col. 9, lines 28-31 . 
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The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a 
whole would have been obvious at the time the invention was made to a person having 
ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

Claims 8, 9, 19, and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Edwards and Moran et al (U.S. Patent 6,525,749), hereinafter Moran. 

Regarding claim 8, Edwards has been shown to teach inputting at least one object on a 
screen display. Edwards further teaches using an input device to draw at least one arrow 
having a configuration recognized by software to associate the arrow with at least one object, 
and having an arrow convey a transaction relating to at least one object. 

Edwards fails to explicitly teach a configuration including a portion of an arrow 
circumscribing at least one object. 

Moran teaches a method for supporting handwritten freeform data similar to the input 
methods of Edwards. Moran further teaches a configuration including a portion of an arrow 
circumscribing at least one object, taught as the "loop-select" of character objects by a user, at 
col. 20, lines 46-49. 

Therefore, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Edwards and Moran before him at the time of the invention to modify the gesture 
recognition system with object manipulation of Edwards to include the "loop-select" feature of 
Moran in order to obtain a gesture recognition system where the user may select objects to be 
manipulated in a lassoing fashion. 

One would be motivated to make such a combination for the advantage of greater 
freeform selection of objects. See Moran, col. 3, lines 38-45. 
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Regarding claim 9, Edwards teaches a further object disposed within a definable 
distance to the head end of said arrow, with a transaction being directed by the arrow to be 
carried out from at least one object to the further object, taught as the use of a drawing of a line 
to "join" segments on a display, at col. 10, lines 36-40. Although Edwards claims a straight line 
as the "join" stroke, drawing a line and drawing an arrow are sufficiently similar in view of the 
claim language to qualify as prior art. 

Regarding claim 19, Moran shows placing at least one object inside a further 
onscreen object disposed within a user definable distance to the front end of at least one arrow, 
taught as the movement of selected character objects in between other character objects or into 
a paragraph object using a move gesture. See Figs. 17(e)-(f) and col. 20, lines 46-49. 

Regarding claim 31, Moran teaches at least one object associated with an arrow 
adapted to change the aesthetic properties of an object after the arrow is drawn, and further 
including the step of touching the object with an input device to implement a transaction, shown 
previously as the selection and removal or relocation of character objects in response to a user 
gesture. 

Claims 20 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Edwards and Naughton et al (U.S. Patent 6,020,881), hereinafter Naughton. 

Edwards has been shown supra to teach implementing a transaction between two 
objects disposed within a user definable distance to the front end of at least one arrow. 
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Edwards fails to explicitly teach such a transaction as directing an electronic signal, and 
redirecting the signal path between objects. 

Naughton teaches a method for interfacing to remote devices that includes gesture 
recognition similar to that of Edwards. Furthermore, Naughton teaches a transaction as 
directing an electronic signal between two on-screen objects, taught as the communication of 
the system with remote devices, at col. 8, lines 50-54. 

Therefore, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Edwards and Naughton before him at the time of the invention to modify the 
gesture recognition system with object manipulation of Edwards to include the remote device 
control of Naughton in order to obtain a freeform object manipulation system capable of 
controlling hardware devices. 

One would be motivated to make such a combination for the advantage of controlling 
multiple devices through one simple interface. 

Claims 27 and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Edwards and Davis et al (U.S. Patent 5,740,436). 

Regarding claim 27, Edwards has been shown to teach the use of arrow gestures to 
control objects on screen, and the use of variable controllers in manipulating objects. 

Edwards fails to explicitly teach the manipulation of a sound file through the use of such 
variable controllers. 

Davis teaches the manipulation of object variables through sliders similar to those taught 
by Edwards. Davis further teaches the manipulation of a sound file through the use of such 
variable controllers, taught as the use of an interface and a volume slider on that interface to 
change variables related to specified sound files, at col. 7, lines 49-55. 
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Therefore, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Edwards and Davis before him at the time of the invention to modify the gesture 
recognition system with object manipulation and variable slider of Edwards to include the sound 
file manipulation of Davis, in order to obtain a method for freeform object manipulation where 
sounds and sound files may be managed. 

One would be motivated to make such a combination for the advantage of easily 
configuring the settings for the sound of a system. See Davis, col. 2, lines 13-19. 

Regarding claim 28, while Davis teaches the manipulation of a sound file by a variable 
controller, and Edwards teaches multiple simultaneous behaviors, both fail to explicitly teach 
providing an interface for the manipulation of both right and left channels in an audio system. 
Such interfaces to audio systems are well-known in the art, allowing for user control of both right 
and left channels. The Examiner takes OFFICIAL NOTICE of these teachings. Therefore, it 
would have been obvious to one of ordinary skill in the art to include an audio interface with 
variable right and left channel settings into the audio control interface of Davis and Edwards to 
provide a complete audio interface. 

Claims 30 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Edwards. 

While Edwards teaches user-drawn arrows to implement transactions in a gesture 
recognition system, variable line styles, and the erasure of such arrows by a user implemented 
gesture, the reference fails to explicitly teach the flickering of arrows after being drawn. Devices 
for drawing the attention to on-screen objects are well-known in the art, and include blinking or 
flickering, highlighting, or emphasis such as bold-faced type or italics. Furthermore, temporary 
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changes to aesthetic properties of on-screen objects are well-known, for example, flashing icons 
in the Microsoft Windows Taskbar to alert a user to a change in the application. The Examiner 
takes OFFICIAL NOTICE of these teachings. Therefore, it would have been obvious to one of 
ordinary skill in the art to apply emphasis to a drawn stroke to alert a user to an object or 
necessary action. 



Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Michael Roswell whose telephone number is (703) 305-5914. The 
examiner can normally be reached on 8:30 - 6:00 M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-31 16. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Conclusion 



Michael Roswell 
5/12/2004 




